Belief Propagation as a Dynamical System: The Linear Case and Open Problems

Systems and control theory have found wide application in the analysis and design of numerical algorithms. We present a discrete-time dynamical system interpretation of an algorithm commonly used in information theory called Belief Propagation. Belief Propagation (BP) is one instance of the so-called Sum-Product Algorithm and arises, e.g., in the context of iterative decoding of Low-Density Parity-Check codes. We review a few known results from information theory in the language of dynamical systems and show that the typically very high dimensional, nonlinear dynamical system corresponding to BP has interesting structural properties. For the linear case we completely characterize the behavior of this dynamical system in terms of its asymptotic input-output map. Finally, we state some of the open problems concerning BP in terms of the dynamical system presented.